Nonparametric Principal Components Regression

In ordinary least squares regression, dimensionality is a sensitive issue. As the number of independent variables approaches the sample size, the least squares algorithm could easily fail, i.e., estimates are not unique or very unstable, (Draper and Smith, 1981). There are several problems usually encountered in modeling high dimensional data, including the difficulty of visualizing the data, slow convergence for models with numerous parameters, bias in variable selection when some important variables are tentatively dropped at some point during the search process, and the problem of multicollinearity that has a number of potential serious effects on the least squares estimates of the regression coefficients (Montgomery and Peck, 1982).